History Through The Presidents’ Words
Chisheng Li
Hierarchical Clustering
The Presidents are clustered based on the similarity of their State of the Union texts using the Jensen–Shannon divergence method. There are approximately 5 groups, and it is apparent that the presidents of similar eras are grouped together.
K-Means Clustering
The Calinski-Harabasz function suggests that 6 is the optimal number of K-Means clusters to group the presidents.
K-Means clustering also shows that the presidents are grouped together by similar eras based on their PCA features.
Word Associations
The following word associations shows how Democratic and Republican presidents differed in their choice of terms in their State of the Union speeches.
Democratic presidents when they discussed “freedom”:
## $freedom
## smashing subjugated speech espousal matching
## 0.22 0.22 0.20 0.17 0.17
## enslaved liberating lords religion likelihood
## 0.13 0.13 0.13 0.13 0.12
## conscription scratch vivid objective world
## 0.11 0.11 0.10 0.09 0.09
## adversaries eternally expression militarism serves
## 0.08 0.08 0.08 0.08 0.08
## translated blessings foes ideological peace
## 0.08 0.07 0.07 0.07 0.07
## reduces fear flags independence undergo
## 0.07 0.06 0.06 0.06 0.06
## wondering
## 0.06Republican presidents when they discussed “freedom”:
## $freedom
## cambodia fighters freedomsour angola defend
## 0.17 0.15 0.14 0.10 0.10
## indivisible interlocked rightfulness speech world
## 0.09 0.09 0.09 0.09 0.09
## burma peace worship america assemble
## 0.08 0.08 0.08 0.07 0.07
## belarus cause champion consign democracy
## 0.07 0.07 0.07 0.07 0.07
## disagree faiths fight free imprisoned
## 0.07 0.07 0.07 0.07 0.07
## individuality planted swelling usa wednesday
## 0.07 0.07 0.07 0.07 0.07
## zimbabwe afghanistan aspirations democratic human
## 0.07 0.06 0.06 0.06 0.06
## individual proudly religious sees spreading
## 0.06 0.06 0.06 0.06 0.06
## tolerance win elections foundations frontiers
## 0.06 0.06 0.05 0.05 0.05
## hate liberty tide values
## 0.05 0.05 0.05 0.05Democratic presidents when they discussed “budget”:
## $budget
## balanced unbalancing cuts pleaded
## 0.26 0.16 0.13 0.12
## fiscal plead ance bal
## 0.11 0.11 0.10 0.10
## balancing deficit octdec octoberdecember
## 0.10 0.10 0.10 0.10
## pricefixing balance billion expenditures
## 0.10 0.09 0.09 0.09
## federal includes antidrug defense
## 0.09 0.08 0.07 0.07
## entitlements fy racketeering requests
## 0.07 0.07 0.07 0.07
## responds spending cash congressional
## 0.07 0.07 0.06 0.06
## ensures expands programs
## 0.06 0.06 0.06Republican presidents when they discussed “budget”:
## $budget
## balanced ravaging spending
## 0.26 0.22 0.14
## expansionary balancing incorporates
## 0.12 0.11 0.11
## jobproducing scheduling spelled
## 0.11 0.11 0.11
## unbalance whittier freeze
## 0.11 0.11 0.10
## trillion billion deficit
## 0.10 0.09 0.09
## submit airpower epa
## 0.09 0.08 0.08
## federal grammrudmanhollings priorities
## 0.08 0.08 0.08
## sets targets budgetary
## 0.08 0.08 0.07
## current fiscal forthcoming
## 0.07 0.07 0.07
## indicates lineitem priority
## 0.07 0.07 0.07
## balance comparable defense
## 0.06 0.06 0.06
## earmark funded modest
## 0.06 0.06 0.06
## spectacle totaling
## 0.06 0.06Democratic presidents when they discussed “energy”:
## $energy
## decontrolled exploratory gasohol unleaded
## 0.38 0.38 0.38 0.38
## windfall solar conservation gas
## 0.38 0.32 0.30 0.28
## alcohol gasoline quadrupled slope
## 0.27 0.27 0.27 0.27
## synthetic clean fuels crude
## 0.27 0.26 0.25 0.24
## renewable incentives dramatically drilling
## 0.24 0.20 0.19 0.19
## rationing atomic gallons households
## 0.17 0.16 0.16 0.16
## enacted fy natural production
## 0.15 0.15 0.15 0.15
## buttressed costimpact currently deregulated
## 0.14 0.14 0.14 0.14
## egyptianisraeli evenhandedly helsinki indigenous
## 0.14 0.14 0.14 0.14
## industrializing objectively premised rebates
## 0.14 0.14 0.14 0.14
## redirection reorientation lowincome sources
## 0.14 0.14 0.13 0.13
## funding increased
## 0.12 0.12Republican presidents when they discussed “energy”:
## $energy
## revitalization atomic cleaner floors technology
## 0.25 0.24 0.19 0.19 0.16
## conservation solar clean atoms enacts
## 0.15 0.15 0.14 0.13 0.13
## geothermal grid petroleum accelerate shortages
## 0.13 0.13 0.12 0.11 0.11
## consuming electricity independence allocation deregulating
## 0.10 0.10 0.10 0.09 0.09
## develop gas generates reliable stockpile
## 0.09 0.09 0.09 0.09 0.09
## wind breakthroughs comprehensive nuclear
## 0.09 0.08 0.08 0.08Democratic presidents when they discussed “security”:
## $security
## social israels medicare beneficiaries collective
## 0.35 0.10 0.10 0.08 0.08
## crediting europes havel lech thai
## 0.08 0.08 0.08 0.08 0.08
## thaicambodian vaclav walesa medicaid afghan
## 0.08 0.08 0.08 0.07 0.06
## aged derives facility health israel
## 0.06 0.06 0.06 0.06 0.06
## livelihood national repassing seniors council
## 0.06 0.06 0.06 0.06 0.05
## enhancing guarantee
## 0.05 0.05Republican presidents when they discussed “security”:
## $security
## social homeland revitalized fbi collective
## 0.25 0.15 0.14 0.12 0.11
## retirement reinforced pundits bioterrorism pacts
## 0.11 0.10 0.09 0.08 0.08
## attachments council dedication doubles medicaid
## 0.07 0.07 0.07 0.07 0.07
## medicare practicing reaffirmed rob survivor
## 0.07 0.07 0.07 0.07 0.07
## unchallenged bipartisan broadened commitments costofliving
## 0.07 0.06 0.06 0.06 0.06
## defense diverted entitlement focused funded
## 0.06 0.06 0.06 0.06 0.06
## peace powerfully priority rd southeast
## 0.06 0.06 0.06 0.06 0.06
## strengthened younger asia boom conserving
## 0.06 0.06 0.05 0.05 0.05
## mutual national strategies vanish
## 0.05 0.05 0.05 0.05Democratic presidents when they discussed “economy”:
## $economy
## global rigid lifeblood deprives
## 0.13 0.13 0.10 0.09
## foreignflag seafaring sixpart straitened
## 0.09 0.09 0.09 0.09
## unbalance usbulk worsening environment
## 0.09 0.09 0.09 0.07
## growing revive simplicity balanceofpayments
## 0.07 0.07 0.07 0.06
## barring combines competitive efficiency
## 0.06 0.06 0.06 0.06
## expanding fashioned frown jobs
## 0.06 0.06 0.06 0.06
## rated recession bust depleted
## 0.06 0.06 0.05 0.05
## dynamic fiber longrun nurture
## 0.05 0.05 0.05 0.05
## shrink stamina stringent strong
## 0.05 0.05 0.05 0.05
## talent
## 0.05Republican presidents when they discussed “economy”:
## $economy
## clamored misdirected parenthetically mismanagement
## 0.20 0.20 0.20 0.16
## practised praised efficiency quartermaster
## 0.14 0.14 0.11 0.11
## expanding growing halftrillion commissary
## 0.10 0.10 0.10 0.09
## jobs healthy strong denounce
## 0.09 0.08 0.08 0.07
## noninflationary competitive constructive expansionary
## 0.07 0.06 0.06 0.06
## grows industrialized peacetime recession
## 0.06 0.06 0.06 0.06
## retrenchment stronger underground wartime
## 0.06 0.06 0.06 0.06
## begins inflationary transition
## 0.05 0.05 0.05Text Hierachical Clustering
Term hierachical clustering shows how frequent terms appeared together in each president’s speeches and highlights the presidents’ policies that were presented to Congress and to the American people.
Word clouds
The following word clouds highlight the frequent terms used by every president and show how the State of the Union speeches evolved by presidency.
George Washington (1790-1796)
John Adams (1797-1800)
Thomas Jefferson (1801-1808)
James Madison (1809-1816)
James Monroe (1817-1824)
John Quincy Adams (1825-1828)
Andrew Jackson (1829-1836)
Martin van Buren (1837-1840)
John Tyler (1841-1844)
James Polk (1845-1848)
Zachary Taylor (1849)
Millard Fillmore (1850-1852)
Franklin Pierce (1853-1856)
James Buchanan (1857-1860)
Abraham Lincoln (1861-1864)
Andrew Johnson (1865-1868)
Ulysses S. Grant (1869-1876)
Rutherford B. Hayes (1877-1880)
Chester A. Arthur (1881-1884)
Grover Cleveland (1885-1888, 1893-1896)
Benjamin Harrison (1889-1892)
William McKinley (1897-1900)
Theodore Roosevelt (1901-1908)
William H. Taft (1909-1912)
Woodrow Wilson (1913-1920)
Warren Harding (1921-1922)
Calvin Coolidge (1923-1928)
Herbert Hoover (1929-1932)
Franklin D. Roosevelt (1934-1945)
Harry S. Truman (1946-1953)
